Modeling the Time to Judge Document Relevance

We want to make timed predictions of human performance by modeling user interaction with a hypothetical user interface. Inherent in making a timed prediction via simulation is knowing how long various actions take. One of the most costly actions a user can take is judging a document for relevance. We present a model of the average time to judge the relevance of a document given the document’s length. We produce two parameterized versions of the model based on two sets of user data. The models explain 26-45% of the variance in the average time to judge document relevance. Our models should allow for more accurate timed predictions of human performance with interactive retrieval systems.